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1. Introduction 


Biometrics have long been used as means to recognize people, mainly in terms of their 
physiological characteristics, for various commercial applications ranging from surveillance 
and access control against potential impostors to smart interfaces (Qazi (2004)) (Xiao (2005)). 
These systems require reliable personal recognition schemes to either confirm or determine 
the identity of an individual requesting their services. The biometric methods, that are usually 
incorporated in such systems, can be categorized to physiological and behavioral (Jain et al. 
(2004)), depending on the type of used features. 

The most popular physiological biometric traits are the fingerprint (Maltoni et al. (2009)) that 
is widely used in law enforcement for identifying criminals, the face (Chang et al. (2005)) 
and the iris (Sun & Tan (2009)). However, despite their high recognition performance, static 
biometrics have been recently overwhelmed by the new generation of biometrics, which tend 
to cast light on more natural ways for recognizing people by analyzing behavioural traits. 
Specifically, behavioural biometrics are related to specific actions and the way that each 
person executes them. In other words, they aim at recognizing livingness, as it is expressed 
by dynamic traits. The most indicative cases of behavioural biometric recognition is gait 
(Goffredo et al. (2009b)), facial expressions (Liu & Chen (2003)) or other activity related, 
habitual traits (Drosou, Ioannidis, Moustakas & Tzovaras (2010)). As a result behavioural 
biometrics have become much more attractive to researchers due to their significant 
recognition potential and their unobtrusive nature. They can potentially allow the continuous 
(on-the-move) authentication or even identification unobtrusively to the subject and become 
part of an Ambient Intelligence (AmI) environment. 

The inferior performance of behavioural biometrics, when compared to the classic 
physiological ones, can be compensated when they are combined in a multimodal biometric 
system. In general, multimodal systems are considered to provide an excellent solution to a 
series of recognition problems. Unimodal systems are more vulnerable to theft attempts, since 
an attacker can easily gain access by stealing or bypassing a single biometric feature. In the 
same concept, they have to contend with a variety of problems, such as noisy data, intraclass 
variations, restricted degrees of freedom, non-universality, spoof attacks, and unacceptable 
error rates, i.e., it is estimated that approximately 3% of the population does not have legible 


www.intechopen.com 


146 Recent Application in Biometrics 


fingerprints (Fairhurst et al. (2003)). Such biometric system may not always meet performance 
requirements, may exclude large numbers of people, and may be vulnerable to everyday 
changes and lesions of the biometric feature. 

In this context, the development of systems that integrate more than one biometrics is an 
emerging trend, since it has been seen that true multimodal biometric systems, that capture 
a number of unrelated biometrics indicators, have significant advantages over unimodal 
ones. Specifically, most of the aforementioned limitations can be addressed by deploying 
multimodal biometric systems that integrate the evidence presented by multiple sources of 
information. A multimodal biometric system uses multiple applications to capture different 
types of biometrics. This allows the integration of two or more types of biometric recognition 
systems, in order to meet stringent performance requirements. Moreover, such systems are 
much more invulnerable to fraudulent technologies, since multiple biometric characteristics 
are more likely to resist against spoof attacks than a single one. 

Last but not least, a major issue of biometric systems is the protection of the sensitive biometric 
data that are stored in the database, so as to prevent unauthorized and malicious use. Given 
the widespread deployment of biometric systems and the wide exposition of personal data, 
public awareness has been raised about security and privacy of the latter. Seemingly, the 
voting of several laws concerning the ethical and privacy issues of private data provide a 
universal solution unless it is accompanied by the appropriate technological tools. 
Unfortunately, simple password-based systems, that provide regular cryptographic solutions 
(Uludag et al. (2004)) can not be easily applied, since the representation of behavioural 
biometric traits is not fixed over time. Thus, the current issue has been confronted with 
modern, sophisticated encryption methods that do not require the exact match of the 
prompted and the original signatures in order to grant access. 


1.1 Related work 

With respect to behavioural biometrics, previous work on human identification can be mainly 
divided in two main categories. a) sensor-based recognition (Junker et al. (2004)) and 
b)vision-based recognition. Recently, research trends have been moving towards the second 
category, due to the obtrusiveness imposed by the sensor-based recognition approaches (Kale 
et al. (n.d.)). Additionally, recent work and efforts on human recognition have shown that the 
human behavior (e.g. extraction of facial dynamics features (Hadid et al. (2007)). However, the 
most known example of behavioural biometrics is the human body shape dynamics (Ioannidis 
et al. (2007) or joints tracking analysis (Goffredo et al. (2009a)) for gait recognition. In the same 
respect, the analysis of dynamic activity-related trajectories (Drosou, Moustakas & Tzovaras 
(2010)) provide the potential of continuous authentication for discriminating people, when 
considering behavioural signals. 

Although there have been already proposed a series of multimodal biometric systems 
concerning static physiological biometric traits (Kumar et al. (2010)) (Sim et al. (2007)) there are 
only a few dealing solely with behavioural traits (Drosou, Ioannidis, Moustakas & Tzovaras 
(2010)). In any case, the main issue in a multimodal biometric system is the optimization 
of its fusion mechanism. In a multimodal biometric system, integration can be done at 
(i) feature level, (ii) matching score level, or (iii) decision level. However, matching score 
level fusion is commonly preferred because matching scores are easily available and contain 
sufficient information to distinguish between a genuine and an impostor case. In this respect, 
a thorough analysis of such score-level fusion methods regarding biometric traits has been 
presented in (Jain et al. (2005)). 
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Since all biometric systems deal with the issue of storing biometric data, different approaches 
regarding their security have been suggested. In the current work, an extension of the 
security template scheme, presented in (Argyropoulos et al. (2009)), is proposed, that bases 
on Error Correcting Codes (ECC) and the modeling of channel statistics. Channel codes have 
been previously used for the development of authentication schemes. Earlier, in (Wadayama 
(2005)), a generic authentication scheme based on channel codes was proposed to improve 
security and prevent unauthorized access in secure environments. Also, in (Davida et al. 
(1998)), a channel coding scheme was presented for secure biometric storage. Error correcting 
codes were employed to tackle the perturbations in the representation of biometric signals and 
classification was based on the Hamming distance between two biometric representations. 
Based on this concept, the fuzzy commitment scheme was introduced to tolerate more 
variation in the biometric characteristics and provide stronger security (Juels & Sudan (2006)). 
In this scheme, the user selects at the enrolment a secret message c. Then, the template consists 
of the difference between the user’s biometric data x and c along c with an encrypted version 
of c. At the authentication, the stored difference d is added to the new biometric representation 
y and y + d is decoded to the nearest codeword c’ using error correcting codes. 

In this respect, a series of encryption methods have been developed to account for the inherent 
variability of biometric signals. Apart from (Davida et al. (1998)), a methodology based on the 
Slepian-Wolf theorem (Slepian & Wolf (1973)) for secure storage biometric via Low-Density 
Parity Check (LDPC) codes was presented in (Martinian et al. (2005)). The multimedia 
authentication problem in the presence of noise was investigated, the theoretical limits of 
the system were identified, and the tradeoff among fidelity, robustness, and security was 
discussed. This approach provides intuition for the proposed method in this paper; the 
biometric recognition problem is considered as the analogous of data transmission over a 
communication channel, which determines the efficiency of the system. Interestingly, the 
problem of coding distributed correlated sources has also attracted much interest in the field 
of video coding recently. The same framework was also employed in (Draper et al. (2007)) 
in order to secure fingerprint biometrics, image authentication (Yao-Chung et al. (2007)) and 
biometric authentication as a wire-tap problem (Cohen & Zemor (2004)). 

In the seminal work of (Pradhan & Ramchandran (2003)), the distributed source coding using 
syndromes scheme was proposed. Based on this work, the field of distributed video coding 
(Girod et al. (2005)) has emerged as a new trend in video coding. Finally, an interesting 
approach of applying the LDPC methodology in multimodal biometric systems has been 
proposed in (Argyropoulos et al. (2009)). 

Similarly to above, one of the major concerns in applications that grant access based on a 
password, a pin or a token, is the protection of the original data to prevent malicious use from 
those who try to access them by fraudulent means. Although this problem in such systems 
has been investigated in depth and sophisticated encryption methods have been developed 
(Stallings (2006)), a significant issue remains the possibility of having the password stolen or 
forgotten. Thus, methods which enable a biometric-related key have been proposed (Alvarez 
et al. (2009)). Thus, the required pin is always carried by the user, since it is encoded on 
himself. 


1.2 Contribution 

In the current chapter, a novel framework for activity related authentication in secure 
environments based on distributed source coding principles and automatically extracted 
biometric keys is proposed. The main novelty is the development of an integrated framework 
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that utilizes biometric key based encryption, in order to assist the channel decoding 
process and to boost the system’s recognition performance. It is shown that the proposed 
system increases the security of the stored templates and ensures privacy of personal 
data, while indirectly provides “hybrid” fusion between static and dynamic biometric traits 
towards improved recognition results. Moreover, unobtrusive, multimodal, on-the-move 
biometric authentication is presented and evaluated in a bimodal scenario, which utilizes two 
behavioural traits of the user. Namely, the gait and the activity-related motion trajectories 
of the head and the hands during specific movements which are seen to provide a powerful 
auxiliary biometric trait are inspected in terms of biometric means for user authentication. 


2. Proposed methodology 


The architecture of the proposed biometric recognition framework is illustrated in Figure 1. 
Initially, from the captured gait image sequence, the moving silhouettes are extracted, the 
shadows are removed and the gait cycle is estimated using state-of-art (SoA) algorithms 
(Ioannidis et al. (2007)), (Cucchiara et al. (2001)). Using a stereoscopic camera, we detect 
those frames in the sequence, whereby the user is standing and we discard them from those 
where the user is walking. Then the visual hull of the moving silhouette is extracted using 
disparity estimation. Once a view normalization is applied by rotating the silhouette, the 
3D reconstructed silhouettes are denoised via spatiotemporal filtering, in order to improve 
their quality. Finally, two SoA geometric descriptors are extracted based on the sequence Gait 
Energy Image (GEI). 


Fig. 1. System Architecture. 


The gait recognition follows the principle of a model-free, feature-based analysis of the 
extracted human silhouettes, whereby geometric methods implement a robust classifier. In 
the following, the activity-related recognition is performed on the users’ movements while 
they interact with a security panel installed at the end of the corridor. The extracted motion 
trajectories that are used as the user’s biometric traits are classified by a Dynamic Time 
Warping classifier and its result is finally fused with the corresponding gait results at the 
score level towards an overall recognition outcome. 


3. Behavioural biometrics 


As it has already been mentioned, the development a novel biometric recognition method 
or the improvement of current State of Art (SoA) methodologies in this area, are not within 
the scope of the current work. In this context, a set of simple but robust activity-related 
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recognition modules have been utilized in the context of the proposed security framework in 
order to build a behavioural multimodal recognition system, where the proposed enhanced 
security template framework (see Section 4) could be tested and evaluated. 

In particular, the first biometric modality consists of SoA gait recognition methodology 
(Ioannidis et al. (2007)) that bases on features extracted from spatiotemporal gait patterns. 
Similarly, the second modality that has been utilized refers to a novel activity-related concept 
that has been initially proposed in (Drosou, Moustakas, Ioannidis & Tzovaras (2010)) and 
deals with the motion related traits left by the user during the performance of some simple 
activities that are performed on a regular basis. Both aforementioned modalities are not only 
directly related to the users’ physiology, but they are also highly governed by the users’ 
habitual response to external stimuli. Thus, they have been seen to provide significant 
recognition capacity, both as stand-alones, as well as in multimodal recognition systems 
(Drosou, Ioannidis, Moustakas & Tzovaras (2010)). 

For the convenience of the reader, a short functional description of the aforementioned 
modalities is included hereafter. Before presenting the security framework, which is the main 
contribution of the current work, a short description of the utilized biometric modalities is 
included. 


3.1 Gait recognition 

Let the term “gallery” refer to the set of reference sequences, whereas the term “probe” stands 
for the test sequences to be verified or identified, in both presented modalities. 

Initially, the walking human binary silhouette is extracted as described in (Ioannidis et al. 
(2007)). The feature extraction process of the gait sequences is based on the Radial Integration 
Transformation (RIT) and the Circular Integration Transform (CIT) (Ioannidis et al. (2007)), 
but instead of applying those transforms on the binary silhouette sequences themselves, the 
Gait Energy Images (GET) are utilized, which have been proven from one hand to achieve 
remarkable recognition performance and on the other hand to speed up the gait recognition 
(Han et al. (2006)) (Yu et al. (2010)). 

Given the extracted binary gait silhouette images I’ and each gait cycles k, the gray level (GET) 
(Figure 2) is defined over a gait cycle as: 


CycleEnd 
GEk=—: Yo I'(k) (1) 
L k=CycleStart 
where C; is the length of the gait cycle and k refer to the gait cycles extracted in the current 
gait image sequence. 


Fig. 2. Gait Energy Images from several users. 


The RIT and CIT transforms are applied on the GEI, in order to construct the gait template 
for each user, as shown in Figure 3 in according to the following equations: 
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RIT seo) m Xo + u cos 0, yo + u sin 0 )du (2) 


where u is the distance from the starting point (xọ, yo). 


J 
RIT(tA@) GEN ro + jAu - cos(tA0), yo + jAu - sin(tA0)) 


=a 
J (3) 


for t = 1,...,T with T = 360° / A0 


for t = 1,...,T with T = 360° / A0, where A0 and Au are the constant step sizes of the distance 
u and angle 0 and J is the number of the pixels that coincide with the line that has orientation 
R and are positioned between the center of gravity of the silhouette and the end of the image 
in that direction. 


y? Integration Path y* Integration Path 


Paz 
ee 
á dx 


Fig. 3. Applying the RIT (left) and CIT (right) transforms on a Gait Energy Image using the 
Center of Gravity as its origin. 


Similarly, CIT is defined as the integral of a function f(x,y) along a circle curve h(po) with 
center (x9, Yo) and radius p. The CIT is computed using the following equation: 


CIT) = f f(xo + pcos0 + p sin 8)du (4) 


where du is the arc length over the path ii integration and 6 is the corresponding angle. 
The center of the silhouette is again used as the origin for the CIT. The discrete form of the CIT 
transform is used, as depicted graphically in Figure 3/right. 


CIT(kAp) = L GEI (xo + kAp - cos(tA0), yo + kAp - sin (tA0)) (5) 


for k = 1,..,KwithT = 360°/A@, where Ap, and A0 are the constant step sizes of the 
radius and angle variables and finally KAp is the radius of the smallest circle that encloses 
the grayscaled GEI (Figure 2). 

The extracted RIT and CIT feature vectors are then concatenated, in order to form a single 1D 
biometric trait. 
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3.1.1 Matching 
The comparison between the number of gallery Gggg and probe Pez; gait cycles for a specific 
feature E € {RIT, KRM} is performed through the dissimilarity score dp. 


dg = min (list = s/ |) Vi, ji € [1, Ggez) and j € [1, Paer) (6) 
whereby ||- || is the Lz-norm between the s] and s?” values of the corresponding extracted 
feature (i.e. RIT & CIT) for the gallery and the probe collections, respectively. 


3.2 Activity-related recognition 

The proposed framework extends the applicability of activity-related biometric traits 
(Drosou, Moustakas, Ioannidis & Tzovaras (2010)), and investigates their feasibility in user 
authentication applications. 

In (Kale et al. (2002)) and (Drosou, Moustakas & Tzovaras (2010)), it is claimed that the traits 
of a subject’s movements during an activity that involves reaching and interacting with an 
environmental object can be very characteristic for recognition of his/her identity. Indeed, 
given the major or minor physiological differences between users’ bodies in combination 
with their individual inherent behavioural or habitual way of moving and acting it has been 
reported that there is increased authentication potential in common everyday activities such 
as answering a phone call, etc. 

In the following, an improved activity-related recognition framework is proposed, that 
employs a novel method for the normalization of the trajectories of the user’s tracked points 
of interest. The proposed algorithm also introduces a warping method that compensates 
for small displacements of the environmental objects and has no effect on the behavioural 
information of the movement at all. 

As of today, activity related biometrics, where the activity is associated with reaching and 
interacting with objects, have always assumed a fixed environment (Drosou, Moustakas & 
Tzovaras (2010)), which is not always the case in real life scenarios. However, significant 
performance degradations can be observed due to the small variances in the interaction 
setting, which are introduced by the arbitrary position of the environmental objects in respect 
to the user at each trial. Thus, a post-processing algorithm towards the improvement of the 
overall authentication performance that can be employed into biometric systems which utilize 
the reaching and interacting concept, is presented in the following. 


3.2.1 Motion trajectory extraction 

The core of the proposed authentication system used on dynamic motion tracking (4f) is 
extensively described in (Drosou, Moustakas, Ioannidis & Tzovaras (2010)) and is briefly 
described in the following so as to make the paper self-contained. The userSs movements 
are recorded by a stereo camera and the raw captured images are processed, in order to track 
the users head and hands via the successive application of filtering masks on the captured 
image. 

Specifically, a skin-colour mask (Gomez & Morales (2002)) (4a) combined with a motion-mask 
(Bobick & Davis (2001)) (Figure 4d) can provide the location of the palms, while the head 
can be accurately tracked via a combination of a head detection algorithm (Viola & Jones 
(2004)) enhanced by a mean-shift object tracking algorithm (Ramesh & Meer (2000)) (4b). 
Given the pre-calibrated set of CCD sensors mounted on the stereo camera, the real 3D 
information can be easily calculated first by performing disparity estimation (4c) from the 


www.intechopen.com 


152 Recent Application in Biometrics 


Bm OOO! 


Fig. 4. Tracking methodology: a) Skin filtering - b) Head Tracking - c) Disparity image - d) 
Motion Detection - e) Possible hands’ locations - f) Motion trajectories. 


input stereoscopic image sequence and then by mapping the 2.5D information onto the 3D 
space. After post-processing (Drosou, Moustakas, Ioannidis & Tzovaras (2010)) that is applied 
on the raw tracked points, based on moving average window and Kalman filtering, equally 
sized and smooth 3D motion trajectories are extracted (Figure 5), which are then used as 
activity related biometric traits for proposed modality. 

A motion trajectory for a certain limb / (head or palms) is considered as a 3D N-tuple vector 
s}(t) = (x(t), y1 (t), zı (t)) that corresponds to the x,y,z-axes location of limbs center of gravity 
at each time instance t of an N — frame sequence. The x,y and z data of the trajectories sj, 
are concatenated into a single vector and all vectors, produced by the limbs that take part ina 
specific activity c form the trajectory matrix S,. Each repetition of the same activity by a user 
creates a new matrix. Both gallery and probe user-specific set of matrices are subsequently 
used as input to the Dynamic Time Warping (DTW) algorithm 3.2.2 that has been utilized as 
classifier for the current biometric modality, in order to provide an authentication score with 
respect to the claimed ID (gallery). 


3.2.2 Matching via DTW 

DTW is used for calculating a metric about the dissimilarity between two (feature) vectors. 
It is based on the difference cost that is associated with the matching path computed via 
dynamic programming, namely the Dynamic Time Warping (DTW) algorithm. The DTW 
algorithm can provide either a valuable tool for stretching, compressing or aligning time 
shifted signals (Sakoe & Chiba (1990)) or a metric for the similarity between two vectors 
(Miguel-Hurtado et al. (2008)). Specifically, it has been widely used in a series of matching 
problems, varying from speech processing (Sakoe & Chiba (1990)) to biometric recognition 
applications (Boulgouris et al. (2004)). The matching between the two vectors is done and a 
path is found using a rectangular grid (Figure 6). 
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Fig. 5. 3D Motion Trajectories extracted during a “Phone Conversation” activity. 


A short description of the functionality of DTW algorithm for comparing two 
one-dimensional vectors (probe & gallery signal) is presented below: 

The probe vector p of length L is aligned along the X-axis while the gallery vector g of length 
L’ is aligned along the Y-axis of a rectangular grid respectively. In our case L = L’ as a result 
of the preprocessing steps (Section 3.2.1). Each node (i,j) on the grid represents a match of the 
iz, element of p with the jy element of g. The matching values of each p(i),g(j) pair are stored 
in a cost matrix Cy associated with the grid. c(1,1) = 0 by definition and all warping paths 
are a concatenation of nodes starting from node (1,1) to node (L, L). 

The main task is to find the path for which the least cost is associated. Thus the difference cost 
between the two feature vectors is provided. In this respect, let (y1 (k), y2(k)) represent a node 
on a warping path at the instance t of matching. The full cost D(y1, y2) associated to a path 
starting from node (1,1) and ending at node (y;(K),y2(K)) can be calculated as: 


k 
D(y1,y2) = D(yi(k — 1), y2(k — 1)) + (y1,y2) = YO c(yi(m), y2(m)) (7) 


m=1 
Accordingly, the problem of finding the optimal path can be reduced to finding this sequence 
of nodes (y1 (k), yo(k)), which minimizes D (y1 (k), y2(k)) along the complete path. 


Fig. 6. Dynamic Time Warping Grid. 
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As stated by Sakoe and Chiba in (Sakoe & Chiba (1990)), a good path is unlikely to wander 
very far from the diagonal. Thus, the path with minimum difference cost, would be the one 
that draws the thinnest surface around the diagonal as shown by the dashed lines in Figure 6. 
In the ideal case of perfect matching between two identical vectors, the area of the drawn 
surface would be eliminated. The size of the closed area S4 around the diagonal can be 
calculated by counting the nodes V(p;,q;) between the path and the diagonal at every row 
(Jayadevan et al. (2009)) as indicated by the following equation. 


1,if (i >j) of N(pi,qj) 

for j = j,j+1,...,j +d, where d =i —j 
1, if (i < j) of N(p;,q;) 

fori =i,i+1,...,i+d,,where d =i-j 
1, if (i = j) of N(pj,qj) 
0, otherwise 


V (pi qj) = (8) 


Thus, the value V(p;,q;) = 1 to these nodes. On the contrary, all other nodes lying outside 
the closed area will be assigned the value V(p;,q;) = 0. Then, the total area S 4 created by the 
path is mathematically stated as following: 


Sa = }, dV (pig) (9) 


whereby 
Finally the total dissimilarity measure DmĮ between vector p and g (Equation 9) can be 
computed as the product of area size S¢ and the minimum full cost D(L, L) (Equation 7): 


Dm = Sa: Dmin(L, L) (10) 


4. Biometric template security architecture 


As far as the security of the biometric data is regarded, multiple feature descriptors from the 
gait modality and the activity-related modality are initially encoded via a Low Density Parity 
Check (LDPC) encoder. In the following, the parity bits of the activity-related modality are 
encrypted via a biometric-dependent key, so that double secured, non-sensitive biometric data 
is stored in the database or in smart cards, which are useless to any potential attackers of the 
system. 

The proposed method, which resembles a channel coding problem with noisy side 
information at the decoder, is shown to improve the authentication rates as they are provided 
from the modality-specific classifiers. Additionally to the already known key-encryption 
methodologies, the encryption of the parity bits of the second modality takes place before their 
storage to the database. The novelty lies in the fact the personal encryption/decryption key 
used, is inherent in the biometric trait of the first modality and thus, it remains unknown even 
to each user. Specifically, in the implemented scenario the biometric key is selected according 
to the height and the stride length of the user. 

The architecture (Figure 7) of the proposed security is thoroughly described in the next two 
Sections, whereby a functional analysis of the utilized distinct components is provided. 
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Gait 
modality 


» 
activity 


Fig. 7. Security subsystem Architecture. 


4.1 Encoding scheme 

The first step towards biometric template protection in the current multimodal biometric 
cryptosystem is based on distributed source coding principles and formulates biometric 
authentication as a channel coding problem with noisy side information at the decoder, as 
presented in (Argyropoulos et al. (2009)). The main idea lies on the fact that perturbations 
in the representation of the biometric features at different times can be modelled as a 
noisy channel, which corrupts the original signal. Thus, the enrolment and authentication 
procedures of a biometric system are considered as the encoding and decoding stages of 
a communication system, respectively. The proposed formulation enables the exploitation 
of the Slepian-Wolf theorem to identify the theoretical limits of the system and minimize 
the size of the templates. Moreover, casting the problem of biometric authentication as a 
communication problem allows the use of well known techniques in communication systems 
such as the exploitation of correlation (or noise) channel statistics by integrating them in the 
soft decoding process of the channel decoder. 

The architecture of the multimodal biometric authentication system is included in Figure 7. 
At the enrolment stage, the feature vectors Foqi and Factivity from the Gait and the Activity - 
related modality are initially extracted as described in the previous section. Then, the extracted 
feature vectors are quantized and encoded using an (n, k) LDPC channel encoder. It must be 
stressed that the rate of the LDPC encoders in Figure 7 is different for each modality and is 
calculated according to the Slepian-Wolf theorem 


Rx > H(X|Y) Ry > H(Y|X) Rx + Ry > H(X,Y) (11) 


where Rx and Ry the achievable rates, H(X|Y) and H(Y|X) are the conditional entropies and 
H(X,Y) is the joint entropy of X and Y gallery and probe feature vectors, respectively. 

The resulting codewords Sgait and Sactivity Comprise the biometric templates of the suggested 
modalities and are stored to the database of the system. Thus, if the database of the biometric 
system is attacked, the attacker can not access the original raw biometric data or their 
corresponding features but only S¢,it and Sactivity, Which are not capable of revealing sensitive 
information about the users. 


Similarly the gait and activity-related feature vectors Fpi and F’ are extracted and 


ctivity 
quantized at the authentication stage. Subsequently, the syndromes Sgait and Sactivity Which 


correspond to the claimed ID are retrieved from the database and are fed to the LDPC 
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decoders. A similar multimodal approach is thoroughly described in (Argyropoulos et al. 
(2009)). Thereby, two biometric traits, i.e. face characteristics and face, have been combined 
via concatenation of their feature vectors. Specifically, once the first modality was successfully 
decoded, the decoded feature vector was concatenated to probe feature vector of the second 
modality. The full feature vector was fed to a second decoder. Thus, enhanced security was 
offered, since the second decoder requires that both feature vector resembles the gallery input. 
In the proposed approach, the system deals with two behavioural biometric traits separately, 
as far as the LDPC algorithm is regarded. However, it must be noted that the two biometric 
templates in the proposed scheme are not secured independently from each other. 

The basic guidelines of the LDPC encoding/decoding scheme will be presented below in 
short, in order to provide a self-consistent paper. 

Given the unimodal protection scheme had been used for every biometric modality 
independently the rate required to code each feature vector. This in turn would affect the 
size of the templates and the performance of the system. 

Even if liveness detection is out of the scope of the paper, the multimodal framework provides 
tools to guarantee that even if the user is wearing a mask, in order to fake the system, he/she 
should also mimic the gait modality. Thus, we are not proposing a solution that will support 
liveness detection at the sensor level, however, we can support security at the signal level due 
to the multimodal nature of the proposed framework. 

Initially, at the enrolment stage, the biometric signatures of an individual for gait and 
activityrelated modalities are obtained. The extracted features form the vector F; = [f1,..., fkl, 
whereby i € gait, activity related and f; € RX. The feature vector F; is then uniformly 
transformed from the continuous to the discrete domain of 2} levels through the function 
u : RÉ => Q* whereby Q = 0,1...,/ — 1. Each one of the resulting vectors q = u(F;) is fed 
to the Slepian.Wolf encoder, which performs the mapping e : QF — C” where C = {0,1} 
outputs the codeword c = e(q),c € C”. 

As already mentioned, herein the Slepian-Wolf algorithm has been implemented by a 
systematic LDPC encoder (Gallager (1963)) (see Figure 8). LDPC codes were selected due 
to their excellent error detecting and correcting capabilities. They also provide near-capacity 
performance over a large range of channels while simultaneously admitting implementable 
decoders. An LDPC code (n,k) is a linear block code of codeword length n and information 
block length k which is defined by a sparse (n — k) x n parity matrix H, where n — k denotes 
the parity bits produced by the encoder. The code rate is defined as r = k/n. A code is a 
systematic code if every codeword consists of the original k — bit information vector followed 
by (n — k) parity-bits. In the proposed system, the joint bit-plane encoding scheme of (Girod 
et al. (2005)) was employed to avoid encoding and storing the L bit-planes of the vector q 
separately. 
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Fig. 8. Encoding via a Parity Check Matrix. 
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Subsequently, the k systematic bits of the codeword ci are discarded and only the syndrome 
s, that is the n — k parity bits of the codeword c, is stored to the biometric database. Thus, the 
biometric templates of an enrolled user consist of the syndromes s = [cx44.--Cn],8 € C(n—k), 
and their size is n — k. It must be stressed that the rate of the two LDPC encoders is different 
because the statistical properties of the two modalities are different. 

Similarly to the enrollment procedure the biometric feature vector F; is obtained quantized 
at the authentication stage. This, together with encoded syndrome g roupi are fed to the 


LDPC decoder. The decoding function d : Cln — k) x RE — Q* combines F; with the 
corresponding syndromes which are retrieved from the biometric database and correspond 
to the claimed identity I. The decoder employs belief-propagation (Ryan (n.d.)) to decode the 
received codewords. 

If the errors introduced in the side information with respect to the originally encoded signal 
are within the error correcting capabilities of the channel decoder then the correct codeword 
is output after an experimentally set (N-=30) number of iterations and the transaction is 
considered as a client transaction. To detect whether a codeword is correctly decoded we 
add 16 Cyclic Redundancy Check (CRC) bits at the beginning of the feature vector Fj. By 
examining these bits the integrity of the original data is detected. If the codeword is correctly 
decoded, then the transaction is considered as genuine. Otherwise, if the decoder can not 
decode the codeword (N;iter > Nc) a special symbol Ø is output and the transaction is 
considered as an impostor transaction. 

From the above, it is obvious that the level of security and the performance of the system 
significantly bases on the number of the parity bits in syndrome s;, apart from the error 
correcting performance of the channel code. 

On the one hand, a channel code with low code rate exhibits high error correcting capabilities, 
which results in the decoding of very noisy signals. This means, that the channel decoder will 
be able to decode the codeword even if the noise in the biometric signal has been induced by 
impostors. Additionally, will consist of many bits and will be more difficult to forge. On the 
other hand, channel codes of high code rate exhibit limited error-correcting capabilities and 
reduce the security of the system since the parity bits produced by the channel encoder consist 
of a few bits. Thus, the design of an effective biometric system based on the channel codes 
involves the careful selection of the channel code rate to achieve the optimal trade-off between 
performance and security. In this respect, a method for further securing the syndrome s; 
is proposed in the following section (4.2). Thus, both the security of a long syndrome is 
preserved, while improved performance is provided. 


4.2 Encryption scheme 

The second phase of the security template algorithm, that is implemented via an encryption 
algorithm (“Keygenerator” box in Figure 7) has a dual mission. On the one hand, it further 
ensures the security of the stored biometric syndromes Sgait and Sactivity (see Section 4.1) and 
on the other hand, it provides a novel method for fusing static physiological information with 
dynamic behavioural traits. An interesting novelty introduced by the specific methodology 
is that the user is no longer obliged to memorize a pin, in order to secure his data. On the 
contrary, the personal password is automatically extracted from a series of N, soft biometric 
features. Thus, the password can neither be stolen nor copied. The utilized methodology is 
presented below. 

In the current implementation of the proposed framework N, = 2 soft biometric 
characteristics have been included. However, the framework can be easily extended to 
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any arbitrary number of soft biometric features, depending on the utilized modalities. In 
particular, the Height and the StrideLength (see Figure 9) of the user have been utilized hereby 
according to the following extendable methodology. 


(a) 


Fig. 9. Stride length (left) and Height (Right) of a user drawn on his/her Gait Energy Image 
(GEI). 


It has been experimentally noticed that the measurement regarding the user’s Stride are much 
more noisy than the ones for his/her Height. Thus, the ratio of aoe has been preferred 
over pure Stride scores, in order to provide a more uniform score distribution for the second 
soft-biometric trait. 

First, a two dimensional hash table is formed, whereby its dimension is limited by the 
minimum and maximum expected value of each soft biometric trait, as illustrated in Figure 


10/left. The resolution f! eight and stride of the grid in each dimension respectively is scalable, 
in order to be optimally adjusted to the needs of each implementation of the framework (see 
Section 6). Thereafter, a unique biometric key is estimated for each cell on the grid (or cube or 
hypercube in the case of N, > 2), according to the corresponding Soft Biometric values. Thus, 
we can write for the general case of N, available soft biometric traits 


Da ni 
Np 
whereby n; stands for the index of the hash table (see Figure 10/left) and is calculated as 

n; = int(%). vi stands for the extracted value of the it" Soft Biometric trait. 

In this context, it is expected that the same user will always obtain the same biometric key, 
since his soft biometric properties will always assign his identity with the same hypercube in 
the grid. 

In the following, the syndromes S; of the i‘h modality are encrypted using the Rijndael 
implementation (Daemen & Rijmen (1999)) of the Advanced Encryption Standard (AES). 
Specifically, the 128 — bit extracted key is used to shuffle the syndrome bits. Simple 
zero-padding technique is performed on the syndrome bits vector, in the cases where their 
number is not a whole multiplier of 2’ bits. Similarly, a 256 — bit key could have been 
extracted, however it has been experimentally seen that it offers a bad trade-off between 
computational resources and security improvement. 

In this respect, the biometric key is used to shuffle/deshuffle the syndromes for the claimed 
ID in the enrollment/authentication phase of the biometric system, respectively. It is easy 
to understand that most probably an impostor would be assigned to a different cell on the 
grid, given his different soft biometric characteristics with respect to the claimed ID. Thus, the 


Key(n1,n2,...,NN,) = (12) 
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Fig. 10. 2D Soft Biometric Grid (Left) - 3D Soft Biometric Grid (Right) 


requested syndrome bits will be wrongly decrypted and the applied dynamic biometric trait 
will never match the decoded one (see Section 4.1). 


5. Score fusion 


In order to provide an overall distance score between the user requesting access and the 
corresponding claimed ID, a fusion of the partial matching distances for each modality has 
to be performed. The fusion approach that has been utilized for the current biometric system 
is based on score level fusion. Thus, the optimal fusion score, that would combine unequally 
amounts of information from each RP is defined as follows 


N 

Stot =W -S = $ wis; = W181 + W282 +... +WNSN (13) 
j=l 

whereby W is the weight coefficient matrix with N w; elements and Sj the score matrix having 
as elements the corresponding partial matching distances s;. 

In this respect, the most common problem that has to be solved in a score-level fusion 
approach is the optimal estimation of matrix W. Given the general structure of a multimodal 
biometric system, it is expected that the authentication capacity would be higher for 
some modalities than for some others. Thus, a rational way for defining the partial 
weight coefficients w; for each modality is to assign a value proportional to their overall 
authentication performance, as follows: 


i EER; ae 
wW; = S S; 
J N 

DN, EER; 


where EER; stands for the Equal Error Rate score for the j!” modality. 
For the current bi-modal (N = 2) biometric system, the values for each w j are defined as: 


EER, EER) 


“1 FER) 4EERS LER ao ERS (a) 
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In order to provide normalized scores in the range of values for each modality, all scores have 
been normalized to a common basis according to the following equation: 


0.5 
TL 


grorm Z ( 


Jel mir ) (16) 


norm 


where sọ is the normalized score value, sz; the non-normalized score, 7 the maximum 


possible score value and Tą an experimentally set threshold for the k!” modality; k € 
{RIT,CIT, DTW}. 


6. Results 


The current Section starts with a short description of the database on which the experiments 
have been carried out. In the following, the identification and authentication results of the 
presented framework implementation are exhibited and qualitatively evaluated. A short 
discussion about the proposed framework is also included. 


6.1 Database 

The evaluation of the proposed secure multimodal biometric framework has been performed 
on the proprietary ACTIBIO-dataset (ACTIBIO ICT STREP (2008)). The current annotated 
database was captured in an ambient intelligence indoor environment and consists of 29 
subjects, performing a series of everyday workplace & office activities. The workplace 
recordings include 29 subjects walking in various paths of 6m, while being recorded by a 
stereoscopic camera that was placed 2.5m away from the walking path and lateral to the 
walking direction. In order to test the permanence of the biometric features, the recordings 
have been repeated in a second session, few months after the first one. 

Regarding the office recordings, the same 29 subjects have been recorded in an ambient 
intelligence (AmI) indoor environment, while they have been performing a series of everyday 
office activities with no special protocol, such as a phone conversation, typing, talking to a 
microphone panel and drinking water. Each subject repeated the same sequence of activities 
8 times in total, split in two sessions while a manual annotation of the database has followed. 
Among the five cameras that have been recording the users from different view-angles, only 
the recordings from a frontal stereoscopic camera have been used for the current work. 
Within the current work, the traits of the aforementioned modalities have been combined, 
in order to create 29 user-specific multimodal signatures. In this respect, each subject has 
been registered to the system (gallery signatures) by using his gait biometric signature together 
with his behavioural response during a phone conversation. Despite the fact that the current 
dataset does also include complicated gait scenarios, whereby the subject is carrying a bag 
or a coat, the simplest version has been utilized within the presented work. Similarly, only 
the “Phone Conversation” activity has been used from the office environment. Similarly, the 
recordings from each modality for a different repetition have been combined in order to form 
the probe signatures for the system. 


6.2 Authentication & verification results 

As it has already been stressed out the major contribution of the current framework is that 
it allows higher level of security of the biometric templates stored in the database, while 
higher recognition performance is simultaneously provide via the encoding of soft biometrics. 
The improved level of security can be easily noticed, when considering that the information 
stored in the database is encrypted. In this respect, not data can be retrieved from the 
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database without providing the correct key. Further, even if this encryption step is somehow 
bypassed, the obtained data remain still of no use to the attacker, since they reveal no biometric 
information as explained in Section 4.1. 

Moreover, in order to illustrate the advances performed in the recognition performance 
via indirect fusion/encoding of the soft biometric traits with the dynamic ones, the initial 
recognition capabilities of the utilized traits is shown in Figure 11. In the same Figures 
the reader can notice a slight degradation in the recognition performance of the activity 
related modality, when the templates that are stored in the database are secured via the 
LDPC encoding algorithm (Section 4.1). Contrary to the 1D feature vector of the gait 
modality, activity related feature vectors are much more complex. Thus, a degradation 
in the authentication performance is more likely due to the noisy errors at the decoding. 
Moreover, a degradation is expected, since this is the trade off for adding enhanced security 
in the biometric system. Specifically, such a deterioration have been mainly caused by the 
unintended reconstruction of an impostor’s feature vector, so that it resembles a genuine user. 
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Fig. 11. EER Scores (left) and Identification performance (right) of the proposed modalities 
prior to encryption. 


As it has been mentioned in Section 4.2 the current framework allows a scalable resolution 
of the hash table that is used for encryption, so that optimal performance of the system is 
achieved, given different soft biometrics. In this respect, the Optimal Functional Point (OFP) of 
the current system has been set according to the results illustrated in Figures 12. Specifically, 
one can notice that an intense degradation of the system’s recognition performance for high 
resolution values (= large number of available keys in Hash Matrix of Figure 10left). This 
is caused by the noisy measurements of the soft biometric trait in different repetitions. For 
instance, let us assume a user that has been registered to the system with a vHeight = 1.79 and 
Ustride = 1.62. He/she would be assigned the key K(17,14). A noisy measurement of his soft 


biometrics at the authentication stage might result that his stored syndrome s was attempted 


to be decoded by a different key Key(n?”™ nbn’) # K(17,14). Thus, the decrypting would 


never be successful and the recognition would fail. The reason for which the EER scores 
of the activity related modality exhibits more fluctuations than the one of the gait bases on 
the following fact: The soft biometric measurements of some impostors in the authentication 
stage did not only lie within the same hash bin as the client, but also their activity related traits 
managed to be decoded via LDPC using the syndrome of the claimed user’s ID. 

In this concept, it can be concluded that high resolution values, which refer to a big number 
of bins in the hash table, are intolerant to noisy measurements. On the other hand, small 
resolution values may result to the fact that all subjects are assigned to the same key K and 
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Fig. 12. EER scores of the proposed modalities as a function of the ss u step. 


thus, the encryption scheme would have no meaning. On the contrary, there is always an 


functional point, whereby the recognition performance of the system is optimal. 


Moreover, the reader can notice in Figure 12 that for the Feige step = 0.25 both modalities 


achieve their authentication performance. Thus, this value can be considered the system’s 
optimal functional point for a given Height resolution in the hash table, experimentally set at 
foes 
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Fig. 13. Client/Impostor Distributions for the gait modality at the optimal functional 
point(left) and without Encryption(right) via the Biometric key. 
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Although there seems to be only small changes in the EER scores for small resolution values, 
the distribution of the genuine/impostor scores significantly changes (see Figure 13). 


0 _ - _ 100- 
— aew 
x w r = A Pa Come Ore) o 
haet Moaste a Orr 

x ad 

oy 
i j 

° i 

© 

| ” 
5“ 
E m 

x 

n 7 

+ Pean w 

a 
y ” EJ » OR C L e s =o Fi ; C g3 » = % 
F ase Reyection Rate Rato 


Fig. 14. EER Scores (left) and Identification performance (right) of the proposed system. 
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Given the optimal functional point of the encryption system, optimal fusion of the soft 
biometrics with the dynamic traits has been also achieved. In this respect, the current test case 
of the proposed framework has been evaluated in terms of its overall recognition performance, 
by performing fusion between the two utilized behavioral biometrics (Sections 3.1 & 3.2) as 
described in Section 5. The derived optimal recognition performance of the bimodal biometric 
system system is illustrated in Figures 14, in terms of both authentication and identification 
capacity. 

Concluding, it must be noted that the potential of the proposed framework in terms of 
recognition performance is significantly high. Given a larger number of soft biometrics, 
an almost 1 — 1 proportion for keys-users can be achieved, which would lead to further 
decreasing of the recognition error. 


7. Conclusions 


Summarizing, the advantages of the proposed method in terms of security and impact on 
matching accuracy for recognition purposes have been thoroughly analyzed and discussed. 
The performance of the proposed method is assessed in the context of ACTIBIO, an EU 
Specific Targeted Research Project, where activity-related and gait biometrics are employed 
in an unobtrusive application scenario for human recognition. The experimental evaluation 
on a multimodal biometric database demonstrates the validity of the proposed framework. 
Most important, the dual scope of the current framework has been illustrated. Specifically, 
the utilization of the encryption algorithm does not only provide enhanced template security; 
it does also provide indirect fusion with soft biometric characteristics and thus it improves 
the recognition potential. Finally, the proposed user-specific biometric key, which exclusively 
depends on the user’s biometry, increases the level of unobtrusiveness of the system, since the 
user is not obliged anymore to memorize pins or to carry ID cards. 
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